Skip to content

feat(webui): preselect the accelerator the machine actually has - #384

Open
hexonal wants to merge 1 commit into
shiyu-coder:masterfrom
hexonal:feat/webui-device-autodetect
Open

feat(webui): preselect the accelerator the machine actually has#384
hexonal wants to merge 1 commit into
shiyu-coder:masterfrom
hexonal:feat/webui-device-autodetect

Conversation

@hexonal

@hexonal hexonal commented Aug 4, 2026

Copy link
Copy Markdown

The device <select> in the webui is authored CPU-first, so the default selection is CPU on every machine — including ones with a working GPU.

Impact

On Apple Silicon this means the demo ships a default that ignores the GPU entirely. Kronos-small inference on 60m bars is roughly an order of magnitude slower on CPU than on MPS, and nothing in the UI hints that a faster device is available — the dropdown looks like a deliberate choice rather than an accident of option ordering. The same applies to a CUDA box where the user does not think to change it.

Why detect server-side

The browser cannot see which backends torch was built against. torch.cuda.is_available() and torch.backends.mps.is_available() are the only honest answers, and they only exist in the Python process. So /api/available-models now also returns recommended_device, and the page applies it when it populates the dropdown.

Behaviour

  • cuda if available, else mps if available, else cpu
  • Falls back to cpu whenever torch is missing or the model library is unavailable, so the endpoint keeps working in the simulated-data mode
  • Only preselects an option that actually exists in the dropdown, so it degrades to the current behaviour rather than setting an unusable value
  • The helper text under the dropdown says which device was detected, so the choice is visible instead of silent

The user can still pick anything they want — this only changes which option starts selected.

设备下拉是 CPU 优先写死的,所以在 Apple Silicon 的 Mac 上,默认选项**静默忽略
了 GPU** —— 用户不动它就是在 CPU 上跑推理,而界面上没有任何提示说本机其实有
MPS 可用。

探测必须放在服务端:浏览器看不到 torch 是按什么后端编译的,只有 Python 进程
知道 `torch.backends.mps.is_available()` 的答案。所以 `/api/available-models`
多返回一个 `recommended_device`,前端拿到后预选对应项,并把选项文本写进提示行
(`detected: ...`),让「为什么选了这个」也可见。

顺序是 cuda -> mps -> cpu。torch 缺失或导入失败时退回 cpu —— 这条路径上
MODEL_AVAILABLE 本来就是 False,页面走的是模拟数据。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant